Skip to content

[MATLAB] Add simple HTTP GET MATLAB Client example#45

Merged
ianmcook merged 5 commits into
mainfrom
matlab
Jan 22, 2025
Merged

[MATLAB] Add simple HTTP GET MATLAB Client example#45
ianmcook merged 5 commits into
mainfrom
matlab

Conversation

@kevingurney

@kevingurney kevingurney commented Jan 22, 2025 •
edited by ianmcook
Loading

Copy link
Copy Markdown
Member

Overview

Now that the MATLAB interface supports reading the Arrow IPC Stream format from a byte array (uint8), we can now easily consume Arrow IPC Streams over HTTP and load them into an arrow.tabular.Table in memory.

This PR adds a simple HTTP GET MATLAB Client example.

This was tested against the Python server. We would be happy to do more testing as needed.

@ianmcook - would you like to take a look at this PR? Thank you!

Example Output from client.m

$ matlab -batch client
Reading Arrow IPC Stream from http://localhost:8008...
DONE ✔
---------------
Results
---------------
Time (s): 6.28
Num Bytes: 3207031800
Num Rows:100000000
Num Columns:4

Notes

  1. We included fairly detailed comments in the example code for completeness. However, if we prefer to keep the client.m script shorter, we are happy to remove these.
  2. The average time it takes to run the client in MATLAB (not including MATLAB startup time) is about 5.5-6s. This example isn't particularly well optimized but does work as expected. README.md instructs users to run client in -batch mode which can result in some additional delay due to MATLAB startup. The script should run fine if MATLAB is launched and then the script is executed interactively, as well. We just thought it would be easier to directly run the script in -batch mode from the command line.
  3. We included some diagnostic output such as number of bytes received and number of rows. However, if there is more information that we want to include, we are happy to add that.
  4. Thanks @sgilmore10 for your help with this pull request!

Closes apache/arrow#40489

@kevingurney kevingurney requested a review from ianmcook January 22, 2025 17:15
@kevingurney kevingurney self-assigned this Jan 22, 2025
ianmcook
ianmcook reviewed Jan 22, 2025
View reviewed changes
Comment thread http/get_simple/matlab/README.md Outdated
ianmcook
ianmcook reviewed Jan 22, 2025
View reviewed changes
Comment thread http/get_simple/matlab/client.m Outdated
@ianmcook

ianmcook commented Jan 22, 2025

Copy link
Copy Markdown
Member

This is excellent; thanks @kevingurney and @sgilmore10!

I wanted to test this myself, so I went ahead and downloaded and installed a trial copy of MATLAB, activated the license, and built and installed the MATLAB Arrow interface by following the instructions in the README. After some fiddling around with missing dylib files and permissions issues (on an Apple Silicon Mac) I got it working and was able to test this against several more of the server examples. Everything worked as expected.

(base) ian@ian-mac % /Applications/MATLAB_R2024b.app/bin/matlab -nodesktop -batch client
WARNING: package sun.awt.X11 not in java.desktop
WARNING: package sun.awt.X11 not in java.desktop

	Trial License -- for use to evaluate programs for possible purchase as an end-user only.

Reading Arrow IPC Stream from http://localhost:8008...
DONE ✔
---------------
Results
---------------
Time (s): 4.95
Num Bytes: 3207031800
Num Rows:100000000
Num Columns:4

Just a couple of suggested edits to the comments and README text.

kevingurney and others added 2 commits January 22, 2025 16:29
@kevingurney @ianmcook
Update http/get_simple/matlab/README.md
5d6c31d 
Co-authored-by: Ian Cook <ianmcook@gmail.com>
@kevingurney @ianmcook
Update http/get_simple/matlab/client.m
479d34f 
Co-authored-by: Ian Cook <ianmcook@gmail.com>
@kevingurney

kevingurney commented Jan 22, 2025 •
edited
Loading

Copy link
Copy Markdown
Member Author

Thanks for the review and for all of the help with testing this @ianmcook! We really appreciate it!

We're very sorry to hear about the dylib and permissions issues! It's not clear to us whether this was because of the MATLAB interface to Arrow or MATLAB in general. Either way, we would love to know more about the issues you ran into here.

If it is convenient / possible for you to reproduce the issue consistently, we would definitely want to fix it.

It sounds like you built the MATLAB interface from source using CMake rather than downloading the MLTBX from the GitHub Releases area? If that's the case, it's possible you might have run into some kind of build infrastructure issue that we haven't encountered before.

@ianmcook

ianmcook commented Jan 22, 2025

Copy link
Copy Markdown
Member

It sounds like you built the MATLAB interface from source using CMake rather than downloading the MLTBX from the GitHub Releases area?

Correct. I sent you an email with the details about the problem I encountered.

@ianmcook ianmcook merged commit 7c03915 into main Jan 22, 2025
@kou kou deleted the matlab branch January 22, 2025 22:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ianmcook ianmcook ianmcook approved these changes

Assignees

@kevingurney kevingurney

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[MATLAB] Create simple example of MATLAB HTTP GET Arrow client

2 participants

@kevingurney @ianmcook